ebab7d0da53c51438720d60c6e2e467c - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Web data integration</title>

</head>

Web data integration

</h1>

</header>

<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr">Web data integration (WDI) is the process of aggregating and managing data from different websites into a single, homogeneous workflow. This process includes data access, transformation, mapping, quality assurance and fusion of data. Data that is sourced and structured from websites is referred to as "web data". WDI is an extension and specialization of <a href="Data_integration" title="Data integration">data integration</a> that views the web as a collection of heterogeneous databases.

Data integration techniques in the context of the web, forms the foundation for businesses taking advantage of data available on the ever-increasing number of publicly-accessible websites.<a href="#cite_note-1">[1]</a> Corporate spending on this area amounted to about USD 2.5bn in 2017, and it is expected that by 2020 the market will reach almost USD 7bn.<a href="#cite_note-2">[2]</a>

<div class="mw-heading mw-heading2"><h2 id="Sources">Sources</h2></div>

Web data integration extends and specializes data integration to see the web as a collection of views of databases accessible over the web protocols, including, but not limited to:<a href="#cite_note-3">[3]</a>

<ul><li>Open data catalogs</li>

<li>Government data catalogs</li>

<li>Web applications and sites

<ul><li>UI (<a href="Web_scraping" title="Web scraping">web scraping</a>)</li>

<li>The semantic web (SPARQL)</li>

<li>HTML embedded structured data</li>

<li>HTML data tables</li>

<li>Spreadsheets</li>

<li>Online encyclopedias</li></ul>

<div class="mw-heading mw-heading2"><h2 id="Data_access_and_transformation">Data access and transformation</h2></div>

WDI has technical challenges different from data integration due to the data access and transformation required for the web data sources being often <a href="Unstructured_data" title="Unstructured data">unstructured</a> or <a href="Semi-structured_data" title="Semi-structured data">semi-structured</a> data without a standard query mechanism.

<div class="mw-heading mw-heading2"><h2 id="Data_quality">Data quality</h2></div>

Understanding the quality and veracity of data is even more important in WDI than in data integration, as the data is generally less implicitly trusted and of lower quality than that which is collected from a trusted source. There are attempts to try to automate a trust rating for web data.<a href="#cite_note-4">[4]</a>

Data quality in data integration can generally happen after data access and transformation, but in WDI quality may need to be monitored as data is collected, due to both the time and the cost of re-collecting the data.<a href="#cite_note-5">[5]</a>

<div class="mw-heading mw-heading2"><h2 id="Applications">Applications</h2></div>

WDI has application in many fields, including bioinformatics,<a href="#cite_note-6">[6]</a> search engines,<a href="#cite_note-7">[7]</a> price comparison,<a href="#cite_note-8">[8]</a> and forensic search<a href="#cite_note-9">[9]</a> data analysis, business intelligence, ecommerce,<a href="#cite_note-10">[10]</a> healthcare, pharmaceutical<a href="#cite_note-11">[11]</a> and product development.

Most price comparison engines and recommendation systems use user generated data to create recommendations for their users. Similarly, healthcare systems use results of competitions conducted on websites like Kaggle<a href="#cite_note-12">[12]</a> to see the accuracy of data and to create user-focused products. In fact, IBM estimates that poor quality WDI is costing companies over $3 trillion<a href="#cite_note-13">[13]</a> in revenue each year.

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.uni-mannheim.de/dws/teaching/course-details/courses-for-master-candidates/ie-670-web-data-integration/">"IE 670 Web Data Integration"</a>. www.uni-mannheim.de. 2019-01-24. Retrieved 2019-02-11.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.opimas.com/research/355/detail/">"Opimas: The Web Data Extraction Market"</a>. Opimas: We begin with an understanding. Retrieved 2019-02-12.</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.webdataintegration.io/wdi/1.0/index.html">"Introduction :: Web Data Integration"</a>. www.webdataintegration.io. Retrieved 2019-02-14.</cite>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite id="CITEREFGiménez-GarcíaThakkarZimmermann2016" class="citation book cs1">Giménez-García, José M.; Thakkar, Harsh; Zimmermann, Antoine (2016). <a rel="nofollow" class="external text" href="https://hal-emse.ccsd.cnrs.fr/emse-01310508/file/JMGFG2016.pdf">"Assessing Trust with PageRank in the Web of Data"</a> (PDF). In Sack, Harald; Rizzo, Giuseppe; Steinmetz, Nadine; Mladenić, Dunja; Auer, Sören; Lange, Christoph (eds.). The Semantic Web. Lecture Notes in Computer Science. Vol. 9989. Springer International Publishing. pp. 293–307. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-319-47602-5_45">10.1007/978-3-319-47602-5_45</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9783319476025</bdi>.</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <a rel="nofollow" class="external text" href="https://metric.vn/">The way to integrate data from E-commerce website platform</a>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://dbs.uni-leipzig.de/en/projekte/DATAINT/index.html">"Web Data Integration"</a>. Database Group Leipzig.</cite>

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.datascienceassn.org/content/web-scale-data-integration-you-can-only-afford-pay-you-go">"Web-scale Data Integration - You Can Only Afford to Pay as You Go"</a>. www.datascienceassn.org. Retrieved 2019-02-12.</cite>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite id="CITEREFSiegelMadnickZhu2008" class="citation journal cs1">Siegel, Michael D.; Madnick, Stuart E.; Zhu, Hongwei (2008). "Enabling global price comparison through semantic integration of web data". International Journal of Electronic Business. 6 (4): 319. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1504%2FIJEB.2008.020672">10.1504/IJEB.2008.020672</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/1721.1%2F40084">1721.1/40084</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:7995576">7995576</a>.</cite>

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.consultancy.uk/news/2840/pwc-buys-kusiri-londonbased-fraud-detection-startup">"PwC buys Kusiri, London-based fraud detection start-up"</a>. www.consultancy.uk. 2015-10-30. Retrieved 2019-02-12.</cite>

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> <cite id="CITEREFOsialKauranenAhmed2017" class="citation book cs1">Osial, P.; Kauranen, K.; Ahmed, E. (April 2017). <a rel="nofollow" class="external text" href="https://ieeexplore.ieee.org/document/7946845/;jsessionid=GoG6IfE94h0FKAxRzD-ndyd2QUyOECVI7Pj9xc2zBt0s7mIuttf_!878820695">"Smartphone recommendation system using web data integration techniques"</a>. 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE). pp. 1–5. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FCCECE.2017.7946845">10.1109/CCECE.2017.7946845</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-5090-5538-8</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:12474147">12474147</a>.</cite>

</li>

<li id="cite_note-11"><a href="#cite_ref-11">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://dataintegrationinfo.com/data-integration-in-healthcare-pharma/">"How Data Integration is Revamping Healthcare and Pharma"</a>. Data Integration Info. 2020-04-27. Retrieved 2020-05-04.</cite>

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.kaggle.com/">"Kaggle: Your Machine Learning and Data Science Community"</a>. www.kaggle.com. Retrieved 2020-05-04.</cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite id="CITEREFImport.io" class="citation web cs1">Import.io. <a rel="nofollow" class="external text" href="https://www.import.io/post/web-data-integration-revolutionizing-the-way-you-work-with-web-data/">"Web Data Integration: Revolutionizing the Way You Work with Web Data"</a>. www.import.io. Retrieved 2020-05-04.</cite>

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2023-12-27" href="https://en.wikipedia.org/wiki/?title=Web_data_integration&oldid=1192002488">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>